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ABSTRACT 


The decreasing Federal budget and increasing demand for quality healthcare 
warrants using cost effective state-of-the-art technology in Department of Defense 
medical treatment facilities. The purpose of this thesis is to compare state-of-the-art 
digital imaging with conventional film-based imaging technology at two Army Medical 
Centers. The baseline activity for the study was Fitzsimons Army Medical Center and the 
digital alternative used was Madiagan Army Medical Center. The Medical Digital Imaging 
Support (MDIS) has been operational at Madigan, the first DoD facility to use the 
equipment, since early 1992. The MDIS system is a network that links several imaging 
modalities within and between medical treatment facilities. This study focuses on the 
physician, the primary user of the system. Physician questionnaire respondents felt that 
digital imaging reduces the amount of retakes, significantly reduces the number of images 
lost, and decreases the amount of time to retrieve an archived image from the storage unit. 
Using the Tri-Tac methodology, the cost effectiveness calculation determines that digital 
imaging technology is 2.54 more effective per capital investment dollar than the traditional 


film-based technology. 
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I. INTRODUCTION 


A. GENERAL BACKGROUND 


As the 21st century approaches, the Government is reforming the national health 
care system. The goals of the reform initiative begin with ‘managed care” and better 
access to health care for citizens. Striving to meet these goals and incorporating many 
lessons learned in the 1980s, the Department of Defense created the ‘Gateway to Care 
Program.” The goal of the program was to give hospital commanders more power to 
treat military health care beneficiaries by creating local physician networks to meet the 
needs of the supported population. Health care in the military has experienced many 
changes in the 1990s and will continue to change even more rapidly in the remaining years 
of the decade. The Department of Defense can provide a testbed to reengineer the health 
care system to provide greater access to care, improved productivity, increased efficiency, 
equity, convenience, and reduced costs. 

The reengineering process begins by inserting information/communication 
technology into the military health care system. The latest initiative in support of that goal 
is called “Telemedicine.” General Gordon R. Sullivan, Chief of Staff of the Army, stated 
in a recent National Forum on Telemedicine that “We project power by leveraging 
telemedicine on the battlefield.” In this sense, telemedicine reduces risk to Service 
members and uses technology to maintain the edge over hostile and threat forces. 

In September 1994, the Assistant Secretary of Defense, Dr. Stephen C. Joseph, 
charged the Army Surgeon General, Lieutenant General Alcide LaNoue with responsibility 
for establishing a plan to develop telemedicine projects to support the National Health 


Care Reform goals and objectives. The vision and over-arching goals of the testbed 


organization are shown in Figure 1-1. 








Telemedicine Principles 


Over-arching Goals: 
Readiness Orientation 
Patient Focus 
Rapid Prototyping 
os Outcome-based Metrics 
a . Open Systems Architecture 
Sound Business Practices 


Telemedicine projects world 
class health care to patients and 
providers anytime, anywhere. 





Figure 1-1. Telemedicine Vision and Over-arching Goals 


The Medical Digital Imaging Support system is a project under the telemedicine 
umbrella that digitizes patient images and sends them via fiber optic cable to a storage 
unit. The computer network links different types of imaging equipment or modalities 
within a facility including: computed radiography, computed topography, digital 
angiography, nuclear medicine, ultrasound, magnetic resonance and digital spot imaging 
(Smith, 1994). Images are stored in a work storage unit and archived images are further 
stored within an optical disk jukebox (See Figure 1-2). There are three levels of data 
storage: the short term which stores images in the system’s random access memory, 
intermediate storage which stores the images on magnetic media, and long term storage 
which stores images on optical disc. The system allows the physician to access stored 
images from the worksite. With film-based imaging technology, physicians would have to 
retrieve images from a storage area often in a remote part of the hospital. The system also 


has a wider range of exposure times, which makes the system more forgiving in producing 

















readable images for the inexperienced technician. The implication is that less images are 
necessary and the system may reduce image retakes necessary with film-based technology. 

In the future, after networks are installed within a facility, they can be linked to 
other satellite facilities using “teleradiology.” The teleradiology capability allows the 
medical treatment facility (MTF) to communicate with other MTFs in a network via fiber 
optic cable or satellite (Cade, 1993). 
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Figure 1-2. The MDIS System Configuration 
The first Department of Defense (DoD) facility to receive the MDIS system was 
Madigan Army Medical Center (MAMC). Madigan is installing the 3rd phase of a 3 phase 
MDIS implementation schedule. As with most new technology systems, there is a learning 
curve associated with using the system and the first operational users experience growing 
pains as they learn to use the system effectively. Several modifications to the software 
interfaces between hospital information systems and between modalities has slowed the 


full implementation of the MDIS system at Madigan. 


3 








The MDIS Project Manager has a vision to modernize hospital imaging systems to 
meet the current and future needs of DoD medical treatment facilities. In support of this 
vision, Lieutenant General Frank F. Ledford, the previous Army Surgeon General said, “I 
am totally committed to making imaging a reality for military medicine in partnership with 
our sister services. We have a chance to design the future and I intend to grasp it” (Cade, 
1993). Georgetown University and University of Washington conducted a Digital 
Imaging Network System (DINS) evaluation project which provided a benchmark in 
picture archiving and communications technology. There were four major conclusions of 
the project as follows: 

e Commercially available digital technology, used under appropriate management 
and with continuous vendor support, can perform the tasks necessary to 
maintain clinical utility. 

e It was possible to communicate, archive, and display images of diagnostic 
quality on a computer monitor. 

e The technology could improve physician efficiency and availability to treat 
more patients. 

e The technology was accepted by a diverse group of referring physicians and 
radiologists. The group included neuroradiology, gastrointestinal, nuclear 
medicine, the intensive care nursery, and radiation medicine.(Crowe, 1991) 

Soon after the evaluation project was completed, in September 1991, the 
Government awarded Loral a $209 million indefinite requirements contract to develop, 
test, and field MDIS to DoD medical treatment facilities (Cade, 1993). To date, the 
Project Office has issued 11 delivery orders for Army and Air Force medical treatment 
facilities against the contract which expires in September 1995. 

Seahawk is a teleradiology project in the Northwest region that will link several 
medical facilities in a joint Service imaging network called the U.S. West Lion Fiber Ring. 
The network consists of Army, Air Force, Navy, and Veterans Administration medical 


treatment facilities and the University of Washington (See Figure 1-3). 
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Figure 1-3. Project Seahawk Network Configuration 


The project received a $10 million appropriation in 1994 and the equipment and 
installation is expected to cost $6.5 million. The project is designed to increase efficiency 
through live consultations via videoteleconferencing and decrease patient referrals to other 
facilities. Cost savings may result by having radiologists consult via telemedicine, 
eliminating contracted radiologists. The project is also expected to improve the quality of 


patient care by providing faster patient services. 


B. PURPOSE 


The purpose of this research is to conduct a cost effectiveness analysis comparing 
the two types of imaging technologies used by two major Army Medical Centers. Much of 
this research will build on previous studies, which captured the costs and benefits of 
digital imaging technology. Ideas for the research project came from the “Futures” 
section of a preliminary payback study conducted by a Government contractor. The 
Futures section presents ideas about potential cost savings from the system. These 
savings need to be measured, validated, and documented. 

The three areas covered in detail in this thesis include the amount of images that 


require retakes, number of lost images, and time it takes for a physician to retrieve an 
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image taken during previous patient visits. Cost effectiveness is measured by the 


technology’s effectiveness relative to the dollars invested. 


C. RESEARCH QUESTIONS 


The primary research question is: “Are there cost effectiveness differences between 
the state-of-the-art digital imaging system and the traditional film-based imaging 
technology used most frequently by Government and civilian medical treatment facilities?” 
This research will address the following subsidiary research questions: 

1. Does digital imaging reduce the amount of retakes compared to traditional film- 

based systems? 

2. Does digital imaging reduce the number of images lost within a medical 

treatment facility? 

3. Does digital imaging improve physician efficiency by reducing the time to 

retrieve images? 


4. What is the effectiveness per investment dollar for both systems? 


D. SCOPE, LIMITATIONS, AND ASSUMPTIONS 


The emerging digital technology has the potential of replacing existing film-based 
technology in civilian as well as military medical treatment facilities. Therefore, it is 
paramount that military leaders make informed decisions about whether to purchase new 
technology. Compared to film-based technology, scientists and medical professionals have 
discovered many applications of digital imaging technology which improve the quality of 
patient care as well as efficiency. For example, teleradiology allows remote diagnostic 
consultation and image transfer support not possible under the old system. With improved 
access to historical patient images, teaching files can also be constructed to train 
radiologists and clinicians to detect disease and illnesses through imaging technology. 

This research focuses on the clinician, who is the primary user of the MDIS 
system. The specific areas addressed include image retake rates, frequency of lost images, 
and time to retrieve archival images. Collected data are analyzed both quantitatively and 
qualitatively using questionnaires and interviews. Since project Seahawk is still in the 
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design phase, the cost effectiveness measure used for this project is relatively limited. The 
MDIS system is functional within Madigan, thus only operations used currently are 
included in this project. The teleradiology applications are excluded from this study. 

A major limitation of this study is that data are only collected at two sites. Since 
Madigan is the first facility to receive the MDIS system, and has the most experience with 
digital imaging, it will be the site for MDIS data collection. The representative major film- 
based medical facility is Fitzsimons. The digital imaging tmplementation is not yet 
complete at Madigan. Therefore, the results of this research only represent a portion of 
the total impact digital imaging technology has on a facility. 

The primary assumption made in this project is that the two sites chosen have 
similar missions, supported populations, radiology protocols, and technical expertise. 
Another assumption is that physician questionnaires at each site do in fact measure how 


the imaging technology is perceived by the system’s primary user. 


E. METHODOLOGY OVERVIEW 


The first step in obtaining information for this paper is a literature search on the 
MDIS system, cost effectiveness analysis techniques, and technical papers about MDIS 
and other Picture Archiving and Communications (PACS) systems. The next step is to 
collect data at two comparable major medical treatment facilities. Madigan and 
Fitzsimons are both Army regional medical facilities, teaching hospitals for military 
physicians, and support a mixed population of active duty military, dependents, and 
retirees. Data were collected using two techniques: questionnaires, and interviews with 
the hospital staff. Personal interviews were conducted with key staff at Madigan and 
Fitzsimons. The information collected in the interviews supplements and verifies the 
questionnaire data to validate the results. Several administrative areas of each facility 
provided cost and workload data. The administrative areas from which data were 


collected include: resource management, patient administration, logistics, radiology, and 


legal staff. 








F. ORGANIZATION OF THE STUDY 


This thesis is divided into five chapters, beginning with the introduction as the first 
chapter. Chapter II provides background information on previous economic studies on 
the MDIS system. Chapter III discusses the methodology used to gather and analyze the 
data. Chapter IV is a detailed analysis of the data. Answers to each of the subsidiary 
research questions are provided within this chapter. Chapter V summarizes the results and 
provides insight to decision makers about the effectiveness of each system in terms of 
dollars invested. This chapter also includes recommendations for future studies 
concerning the cost effectiveness of digital imaging technology. The Appendices include 


the questionnaire instrument, detailed questionnaire results, and a format for interviews. 





IL. BACKGROUND 


The world of radiographic technology has undergone tremendous evolutionary 
change within the last century. One radiology technologist with 30 years experience in the 
field describes this evolution as “glass to glass.” Exactly 100 years ago a German 
physicist named Wilhelm Konrad Roentgen discovered the X-ray for use in diagnostic 
imaging. In the early days, diagnostic images were captured on photographic glass plates 
covered with a thick emulsion. After an image was taken, the emulsion was scraped off 
and the glass plates were used again. Today these images are captured using digital 
processes that capture images behind the glass of a computer monitor, hence the “glass to 
glass” evolution. (Chase, 1994) 

This chapter consists of a review of similar studies that have been performed in the 
past and compares how this research project differs from previous studies. This chapter 
sets up the follow-on chapters that compare the cost effectiveness of a film-based system 


to a digital system. 


A. PRELIMINARY PAYBACK ANALYSIS 


In a preliminary payback analysis, the Project Office contracted with Sherikon to 
verify prior predictions of potential cost benefits from MDIS, based on micro-analyses 
conducted by independent researchers. In the Sherikon study, the costs were broken 
down into first-order effects, second-order effects, and third-order effects based on a 
— Scientific American article by Malone and Rockart (1991). The article describes some of 
the universal benefits seen by organizations which have installed computer networking 
information technology systems into the work place to reduce costs, improve productivity, 
and create innovative ways of doing business. First-order effects are direct costs that are 
avoided as a result of the information technology system. These costs may include 
material and labor. The article describes the first-order effect as, “reducing coordination 
costs by the substitution of information technology for human coordination.” The second- 
order effects describe more intangible benefits such as productivity improvements. Finally, 


third-order effects represent the long term benefits of the system. An example of a third- 
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order effect is innovations that improve products or processes that would not be possible 
without the new information system. 

The analysis calculated a system payback period of 5.1 years after total 
implementation. The net present value for the system was $9.8 million and a benefit to 
cost ratio was 2.19 (Sherikon, 1994). Unfortunately, there are some serious flaws with 
the results. The cost figures for film savings are based on a totally filmless system. 
However, Madigan runs a dual system at the present time, generating both film copies and 
soft-copy images so there are no savings on film. Many of the results reported are 
inaccurate because of this assumption. 

In Sherikon’s payback analysis, the project office hoped to verify prior predictions 
by independent researchers based on data from the Digital Imaging Network System 
project. As more sites receive the MDIS system, more accurate cost and benefit analyses 
can be conducted to reflect the changing costs and benefits associated with MDIS systems 
in DoD medical treatment facilities. In large developmental projects such as MDIS, the 
cost and benefit data evolve as the technology matures. The data collected during this 
research will be used to validate other cost benefit studies. It represents a snap shot in 
time of the evolving digital imaging technology. 

This research project grew out of the Futures section of the Sherikon study. The 
following is the list of future research areas in the Sherikon study: 

e Reading Times 

e Interpretation Times 

e Retake Rates 

e Turnaround Times 


e Physician Acceptance 


Image/Report Availability 


Teleradiology 


e Telemedicine 


Silver Recovery 


Handling and Storage Costs 
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e Paper Processing 

e Lost Images and Reports 

e Water-soluble Contrast Media. 

The italicized bullets were chosen for this project because they are independent of 
the duality (film and digital images) that exists within the Radiology Department at 
Madigan. 


B. ECONOMIC ANALYSIS FOR MDIS AT THE NEW BROOKE ARMY 
MEDICAL CENTER 


An economic analysis was performed by project office personnel to determine the 
best course of action for the new hospital at Brooke Army Medical Center, in San 
Antonio, Texas. The study considered three configurations for the Radiology Department 
in the new hospital facility. They are as follows: 

e Alternative 1: Incorporates the existing MDIS equipment into the new facility 

(a 20% filmless facility), 

e Alternative 2: Move existing equipment to the new hospital and add enhanced 

equipment (an 80% filmless facility), 

e Alternative 3: Use a conventional film-based Radiology Department. 

To select the best alternative, the study analyzed incremental costs, that is the cost 
differences associated with each alternative. The study provides exhaustive cost 
information on film costs, chemical costs, storage costs, and associated equipment. 

The study also reports on the successful implementation of digital imaging 
technology to date. A radiologist at Brooke indicates that digital imaging has improved 
the image loss rate from 20% to less than 1% and decreased the image retake rate from a 
range of 8-12% to less than 5%. The time to retrieve an image from the MDIS system at 
a workstation is 2 to 3 minutes for images archived on optical storage media. (Winters, 
1994) The installation at Brooke is successful in incorporating several modalities, 


including: computed radiography, computed tomography, ultrasound, and angiography. 











This research hoped to obtain similar results when comparing Madigan with Fitzsimons 
Army Medical Centers. | 

The Brooke Army Medical Center analysis found that alternative 1 provided a 
$113,325 cost savings using a 20% filmless operation. Alternative B cost $4 million for 
enhanced equipment but saves $481,630 by using an 85% filmless operation. A film-based 
system, Alternative C, cost $.48 million for reprinting digital images but saved $3.8 million 
in maintenance, utilities, and personnel. However, Alternative C incurred additional costs 
to redesign and reconstruct a Radiology Department in the new facility that could process 
and store film. 

One of the more significant cost benefits of digital imaging discussed in the study 
involved avoided legal costs. The research indicated that, “In the last two years, the San 
Antonio Assistant U.S. Attorney estimates that a total of $1.5 million was paid out in 
settlement or judgment for three Brooke medical malpractice cases resulting from a lost 
diagnostic image.” (Winters, 1994). Consideration was given to other legal claims 
involving lost images at Brooke and the conclusion drawn was that malpractice 
settlements cost the Government approximately $1 million per year. Combining the costs 
with the benefits favors Alternative 2, moving the current MDIS equipment from the old 


facility and adding enhancements to obtain an 85% filmiless Radiology Department. 


C. EVALUATION OF MDIS BASED ON CLINICAL EXPERIENCE 


The first Department of Defense facility to receive the MDIS system was Madigan 
Army Medical Center in March 1992. Madigan is installing the second phase of a three 
phase MDIS implementation schedule. The two articles discussed previously represent 
empirical research in the area of cost benefit and cost effectiveness. The remaining articles 
discussed in this Chapter report on the performance of the system under operational 
conditions. 

An article written by one of the system’s early radiology users reports both good 
and bad lessons learned by the first operational users. On the good side, the system has 


archived more than 375,000 computed radiography (CR) images within the system’s 
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operational life. The MDIS system has improved the medical treatment facility’s ability to 
store and retrieve images. (Smith, 1994) Previous studies indicated that on the average a 
large treatment facility such as Madigan lost up to 38% of the images taken (Leckie, 
1992). 

In addition, fail-safe strategies have been continually improved to maintain 
continuous clinical image availability during times when the MDIS system or components 
malfunction. After two years of using digital imaging, Madigan reports a 1% image loss 
rate, primarily due to computed radiography interface errors or human errors. Similarly, 
ergonomic aspects of the equipment placement were refined with time which improved 
system useability. Furthermore, unique patient cases can be assimilated into teaching files, 
improving the physician training program. Finally, the MDIS system can correct an over 
or under exposed image through electronic window and leveling techniques (adjustment of 
the brightness and contrast on the computer monitor), providing for a diagnosis without 
retaking the image. (Smith, 1994) 

On the downside, the system has not been successful in networking several of the 
other imaging modalities, including magnetic resonance, nuclear medicine, ultrasound, 
angiography, and the cardiac catheterization lab due to software interface problems. 
(Smith, 1994) Another operational problem with the system involves the radiologists’ 
need for hardcopy film. Throughput is the process whereby the system preloads images 
for the radiologists’ diagnosis. At Madigan, the system cannot provide adequate 
throughput of images to keep the radiologists busy enough to meet the demands of the 
workload within the department. However, Brooke Army Medical Center also installed 
the MDIS system and the Radiology Department has almost completely converted to 
softcopy diagnosis. Their Radiology Department has obtained a 20% filmless operation 


and hopes to obtain an 85% filmless operation with additional MDIS equipment (Winters, 
1994). 
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Hi. METHODOLOGY 


A. DEFINING THE PROBLEM 


The purpose of this research is to analyze three measures of effectiveness (MOEs) 
for the MDIS system. The three MOEs or measurement criteria are retakes, physician 
waiting time for archival image retrieval, and number of images lost within a facility. This 
analysis compares measurable information that is available from Madigan Army Medical 
Center for digital imaging technology with a conventional radiology system at Fitzsimons 
Army Medical Center. Many other criteria can be measured in the future as the system’s 


phased installation is completed. 
B. PROJECT SCENARIO 


The MDIS system is an information management system which primarily supports 
the physician in diagnosing and treating diseases which require medical imaging 
modalities. The system digitizes the images and stores them on optical discs for future 
use, eventually eliminating the need for hardcopy film and record storage areas. The 
modalities currently available at Madigan include computed radiography, fluoroscopy, and 
digital spot imaging. The indefinite requirements contract calls for integrating other 
modalities as well, including: magnetic resonance imaging, computed tomography, 
ultrasound, and nuclear medicine. Software problems have slowed the implementation 
process and to date it is unknown when the other modalities will be integrated within the 
system. Hence, this research effort only considers a portion of the true cost information 
(retakes, archival images, lost images). | 

The digital imaging facility used in this research project is Madigan Army Medical 
Center located in Tacoma, Washington. Madigan opened a new hospital at Fort Lewis in 
February 1992. The original hospital, built in 1944, was a sprawling, one story 
cantonement style building designed to prevent the spread of disease and fires. Today the 
hospital supports 99,000 active duty members, their dependents, and military retirees. The 


new facility is equipped with many innovative technologies including: energy efficient 
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skylights, a medical mall (with “one stop shopping for your health care’), a robotic 
medical material transport system, and the medical digital imaging and support (MDIS) 
system. The hospital’s average daily workload consists of approximately 40 surgeries, 
29,500 laboratory procedures, 3,700 perscriptions filled, 2,600 images taken, and 7 babies 
delivered. The staff consists of 1400 civilians, 700 officers (400 physicians), and 800 
enlisted. (Public Affairs Office, 1995) 


c. BASELINE SCENARIO 


The baseline facility used in this research is Fitzsimons Army Medical Center 
located in Aurora Colorado. The medical treatment facility has multiple missions: 
maintain or restore health, train health care providers, provide quality health care, and 
obtain excellence through research. The hospital serves 57,960 patients within a 40 mile 
catchment area. The hospital provides tertiary care and consultant responsibilities in a 14- 
state region. There are 155 military physicians, 22 civilian physicians, and 121 continuing 
medical education physicians. The facility averages 47,000 outpatient visits per month. 
(Public Affairs Office, 1994) 

The radiology department primarily uses conventional radiology imaging 
equipment (film-based technology). They also have a mini-PACs system which enables the 
radiologist to store and retrieve images using a computer system. The Chief, Radiology 
plans to have personal computers modified and networked with the mini-pacs so that 


physicians will be able to view the images at their worksites. 


D. JOINT TACTICAL COMMUNICATIONS (TRI-TAC) METHODOLOGY 


The Joint Tactical Communications (TRI-TAC) Office, located at Fort Monmouth, 
New Jersey, is responsible for the Cost Effectiveness Program which includes system 
planning, trade-offs, testing, economic analyses, and cost analyses involved in the 
acquisition processes of joint tactical communications systems and equipment. They use a 
Cost Effectiveness Program Plan (CEPP) which outlines measures of effectiveness, life 
cycle cost estimating, technology forecasts, communications scenarios, risk analyses, and 
simulation. 
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The CEPP is a three volume document. The first volume provides an overview of 
the scope of the plan, Service and agency relationships, and establishes the responsibilities 
for a joint Service/Agency Cost Effectiveness Coordinating Committee. The second 
volume contains a conceptual model used for this cost effectiveness research project. It 
also provides guidelines for the modification and application of the model to a myriad of 
new technology equipment alternatives. The third volume provides instructions and 
guidance for the computation of life cycle cost estimates. It also provides formats for 
reporting in compliance with Department of Defense financial regulations and policies. 

The Department of Defense often uses the TRI-TAC methodology as a decision 
making tool in evaluating systems. The method yields a number that tells the cost 
effectiveness per dollar of investment cost for multiple MOE assessments. The TRI-TAC 
methodology also uses figures of merit, subjective weighting criteria, an effectiveness 
index, and utility theory. 

System effectiveness can be expressed as one or more figures of merit representing 
the extent to which the system is able to perform the intended function. The figures of 
merit (FOM) chosen for this research project are availability, dependability, capability, and 
useability. Availability refers to the degree the system is operable when required by the 
user to perform a specific mission task or procedure. Availability is a function of 
Operating time (reliability) and downtime (maintainability and  supportability). 
Dependability is the measure of the system operating condition at one or more points 
when needed. Capability and useability are both figures of merits which describe the 
system’s performance parameters. Examples of capability include storage capacity, image 
clarity, and band of acceptable image quality. Useability refers to the ease with which the 
mission can be accomplished with each alternative. Useability interfaces with the human 
factor or system operator. 

As mentioned previously, the three MOEs measured in this research effort include 
retakes, time to retrieve historical images, and lost images. The quality of the image 


technology alternative is a function of each MOE. The function identifying this 


relationship is shown: 











Q=f(R, H, L) 
where, 

R = Retakes 

H = Historical Images 

L = Lost Images. 

Each MOE is given a utility value using Table 3-1. The utility figure reflects the 
relative performance of an alternative with respect to the baseline alternative. Therefore, 
Fitzsimons automatically gets a baseline score for each MOE. A qualitative assessment of 
the digital system performance is made for each MOE based on the utility assignment 
criteria. The utility for the alternative, Madigan, is discussed in greater detail in the 


analysis chapter. 


Barely meets minimum essential requirements 
Less effective than the baseline 


Baseline 


More effective than the baseline 


More effective to the extent that the MOE should be a principal 


consideration in the selection of a preferred alternative. 





Table 3-1. Utility Assignment Criteria 

After the utility values are determined for each MOE, weights are assigned to each 
figure of merit based on its contribution to the system’s performance as a whole. The 
criteria for determining the relative contribution to the system can be subjective. 
Therefore, a sensitivity analysis maybe necessary. When each FOM is weighted for each 
MOE, a matrix of values is constructed to establish the total FOM for each alternative. 
These values are scored subjectively to determine the overall effectiveness value. 

The overall effectiveness score for each alternative is divided by the equipment and 
maintenance costs for each system. The operations costs are not included in this project 
because Madigan, the digital site, is still running a dual system which incorporates many 


practices of the film-based technology system. A net present value calculation for each 


18 

















alternative serves as the denominator when deriving the final units of effectiveness per 
dollar investment for each alternative. 

Every few years the Office of Management and Budget (OMB) publishes a 
directive concerning discount rates to be used in Government cost analyses. For the 
purposes of this study, dollar values are given in nominal terms. Discounted dollars help 
determine the value of future benefits and costs associated with different alternatives. The 
most current OMB Circular Number A-94 dated February, 1994 mandates using 5.5% for 
spread sheet calculations when the time Ronzdn for the net present value calculation is 


seven years. 


E. CONSTRAINTS ON THE PROJECT 


Budgetary: The travel cost to collect data for this research is limited to one trip to 
each facility to establish points of contacts, collect information, and take measurements. 
The total cost of this travel is approximately $1800. | 

Data Collection Sites: Because the MDIS system is new there are only two 
operational sites from which to collect information: Brooke Army Medical Center, and 
Madigan Army Medical Center. Madigan was selected for this study because it has had 


the system the longest of the two facilities. 


F. DATA COLLECTION 


Data was collected primarily via questionnaires and interviews with hospital 
personnel at the two Army Medical Centers. Questionnaires and interviews were chosen 
as the collection instruments because the respondents are the subject matter experts 
regarding imaging technology. Therefore, this research focuses on the primary users of the 
MDIS system, the physician. The research assumes that the two medical treatment 
facilities have similar workload data. 

The dendritic analysis technique is used to decompose each measure of 
effectiveness into their critical components. The dendritic analysis approach (Test and 
Evaluation Guide, 1993) is used to chart the primary, subsidiary, and tertiary research 
questions and the test measurement technique used to answer the questions (Figure 3-1). 
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Does digital imaging reduce the amount of retakes 
when compared to the What are the causes of retakes? —— Interview (Radiology) 
traditional film based system? 

What percent of patient images require 


retakes? ae Questionnaire 


Interview (Radiology Departments) 


Does digital imaging reduce the number of images 
lost within an MTF? What percent of archival images are not. + -—————® Questionnaire 
available when requested by a cliniaan? 


Are there cost What are the consequences of lost images? —-— Independent Research 
effectiveness Army Surgeon Generai's Office 
differences between What are the causes of lost images? ———> Questionnaire 
digital imaging 
and film-based 
imaging systems? 
Does digital imaging improve physician efficiency 
by reducing time to retrieve images? Of the personnel listed below, place a number from 
1 to 5 in the boxes to indicate the person who 
most frequently retrieves images from the image 


storage area? ———> Questionnaire 


On the average, how much time does it take to 
retrieve an image from your organizations record 
storage location? —————> Questionnaire 


What is the total outpatient load ateach = ————}> Biostatistician Interview 
facility? 


What are the annual costs for both systems? 
What are the investment costs? ———-> MDIS Project Office 


What are the maintenance costs? —————> Biomedical Maintenance 


What are the utility scores for each 
alternative? —— —}> Decision Maker Evaluation 


What figure of merits will be used to 

measure system effectiveness? — > Decision Maker Evaluation 
What weights will be assigned to each 

figure of merit? —— > Decision Maker Evaluation 





Figure 3-1. Dendritic Approach to Cost Effectiveness Analysis 


G. QUESTIONNAIRE 


The questionnaire was validated through a pilot test. The questionnaire was used 
to answer the questions in the Dendritic Approach (Figure 3-1). The questionnaire was 
first administered to clinicians in Army Medical Centers other than Madigan and 
Fitzsimons. The pilot test validated the numbers and thresholds in the survey’s multiple 
choice questions. Three Army Medical Centers participated in the pilot test including 
Brooke, Tripler, and Eisenhower and the clinical respondents provided little constructive 


criticism. 
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The questionnaire was administered to physicians who use the Radiology 
Department at each site. Physician opinions concerning their experience with the imaging 
technology present at their facility was sought. The desired sample size was a minimum of 
30 respondents at each medical facility. The Central Limit Theorem provides a rule of 
thumb stating when the sample size (n) is => 30; the sample has approximately a normal 
distribution. In an analysis of variance test, one of the assumptions made is that the 
population or treatment distributions are all: normal with the same variance. By obtaining 
a sample size greater than 30, normality can be assumed by the Central Limit Theorem. 
(Devore, 1991) 

The technique used to select respondents to the questionnaire is a judgmental 
sample. Questionnaires were given to physicians who volunteered their time (see 
Appendix A, sample questionnaire). The questionnaire was administered to physicians 
who worked with the Department of Radiology at the two medical treatment facilities in 
October, 1994. Approximately 100 questionnaires were distributed at each site. At 
Madigan, the response rate was 58 responses out of a pool of 400 (14.5 %). At 
Fitzsimons, the response rate was 36 responses out of 295 (12.2%). The analysis assumes 
that the opinions of the respondents are representative of the population at each site. The 


consolidated questionnaire data collected are shown in Appendix B. 


H. INTERVIEWS 


Interviews with hospital personnel were conducted at the work site after 
completing the questionnaires. The interviews collected specific information for the 
research project that was not available from the questionnaire responses. The information 
validated and extended the information in the questionnaires. Interviews were primarily 
conducted with hospital support staff and remained anonymous if requested; otherwise the 
information used in the study is quoted from the source. Follow-on telephone interviews 


were conducted as needed using the statement in Appendix C. 
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IV. ANALYSIS 


The analytical portion of this research demonstrates both quantitatively and 
qualitatively whether or not the MDIS system is cost effective for the Army based on data 
collected on three measures of effectiveness (MOEs). The three MOEs are: alee: 
physician waiting time for archival image retrieval; and number of images lost within a 
facility. A comparison between two similar medical treatment facilities focusing on 
physician opinions will be the basis for this analysis. First, the three MOEs will be 
analyzed to identify the trends and the associated costs for each. Questionnaire data are 
used in the following paragraphs to answer the subsidiary research questions. Factual 
interview data are used in spreadsheet computations. Next, a modified life cycle cost will 
be computed for each alternative using maintenance and capital investment costs only. 
Finally, units of effectiveness per dollar investment will be determined for each alternative 


using the TRI-TAC Methodology discussed in Chapter II. 


A. DOES DIGITAL IMAGING REDUCE THE AMOUNT OF RETAKES 
WHEN COMPARED TO THE TRADITIONAL FILM-BASED SYSTEM? 


Image retakes occur for a variety of reasons. The major categories for image 
retakes within the Radiology Department at Fitzsimons include over or under exposure, . 
fog film, poor patient positioning, processing error, artifacts on film, machine error, and 
technician error (see Appendix B). At Fitzsimons, poor patient positioning causes about 
50% of the retakes. The second largest category of retakes is over and under exposure of 
the radiographic plates. This accounts for 25% of retakes at Fitzsimons (Leonard, 1994). 
Each of these causes can be improved with the implementation of digital imaging 
technology. Under and over exposed film can be accomodated with window and leveling 
techniques and a diagnosis can be obtained. As clinicians become more proficient with 
this technology, poor patient positioning can be overcome by manipulating the digital 
image. Both poor positioning and under/over exposure of film should result in a 


significant difference between digital imaging and film-based imaging technologies. 


23 





Retake rates at Madigan are captured in a much less rigid manner because some 
sections are filmless and use only softcopy images while the majority of sections still 
produce hard copy film. Softcopy retake data are not captured and are not reflected in the 
figures provided by the Madigan Radiology staff. (Carter, 1994) With the digital imaging 
technology, films can be processed over a wider range of exposure times. This results in a 
more forgiving system than conventional radiology in terms of over and under exposed 
films. With conventional radiography, a natural feedback mechanism exists in which an 
underexposed film is too light whereas an overexposed film is too dark.(Smith, 1994) 

Retake data were collected at the two medical treatment facilities to determine 
whether the digital imaging technology significantly changed the number of retakes. Both 
facilities were able to provide 10 months of retake figures expressed in percentage of 
images taken in the calendar year, 1993. The graph shown below (Figure 4-1) depicts the 
month by month comparison between the two MTFs. The zeros depict months in which 
the data were not available. Madigan did not have figures for October and November due 
to system downtime and Fitzsimons did not have the retake figures for January and 
February. 

An interesting trend appeared in the data provided by Madigan. The digital system 
became operational in the early part of 1992 and retake figures were captured for the last 
6 months in 1992 with a mean retake rate of 7.9%. In 1993, the retake rate began at 5.9% 
and steadily declined to 2%. This suggests a learning curve which over time will show 
significant differences between digital imaging technology and conventional imaging 


technology. 
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Figure 4-1. Retake Analysis for Madigan and Fitzsimons 1993 
A one-way analysis of variance (ANOVA) with a 95% confidence interval was 
conducted to determine if the retake rates at the two different facilities were statistically 
different. The null hypothesis states that there is no difference between the Madigan 
retake rate and the Fitzsimons retake rate as a result of the different imaging technologies. 
The p value is much greater than the .05 value necessary to reject the null hypothesis. 
Therefore, based on the retake rates provided for 1993, there is no significant difference in 
the amount of retakes generated by the different imaging technologies (see Figure 4-2). 
This finding is further substantiated by a questionnaire administered to the physician staff 
at both facilities. The majority, 65% of the MAMC staff and 61% of the FAMC staff felt 
that the retake rate was between 1-10% (see Appendix B). 
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Figure 4-2. ANOVA Comparing Madigan and Fitzsimon Retake Rates 1993 

The data that were collected are not inconsistent with the information stated, if 
consideration is given to the possibility of learning to use the system to its full capability, 
particularly in terms of under/over exposure and poor patient positioning. Figure 4-1 only 
considers a strict average of all monthly retake data. If one considers the learning process 
then the early months skew the true average and the true average is more realistically 
lower when one averages the last six months of data collected in 1993. Validation of this 
theory could best be accomplished by collecting the retake rates at both facilities for 1994. 
However, this information was not available for this project. 

In summary, the results of the questionnaire do not show a significant difference 
between MDIS and film-based technology with regard to retake figures based on a simple 
average. The data clearly show a trend of decreasing retake rates at Madigan, possibly 
due to the learning associated with window and leveling techniques and making a 
diagnosis from an image that was not positioned correctly. More data need to be 


collected to validate this hypothesis. 


B. DOES DIGITAL IMAGING REDUCE THE NUMBER OF IMAGES LOST 
WITHIN AN MTF? 


Under a conventional radiology system, there are many reasons that records are 


lost. For example, both Madigan and Fitzsimons are Department of the Army teaching 
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institutions in which many resident physicians are assigned. Consequently, images are 
frequently retained by physicians to provide future teaching files. Many patients do not 
trust the filing system and check-out images without returning them to the medical 
treatment facility. The frequency of lost images 1s also a function of the size, capacity, and 
population supported by a facility. Often records end up in clinics, physician offices, ward 
administrative areas, or within “black holes.” The final major cause of lost images is 
administrative type errors such as mixing up record jackets, misfiling images, or recording 
incorrect patient data on an image. 

A study conducted in 1992 by Dr. Bob Leckie, radiologist, indicated that before 
the MDIS system was installed at Madigan up to 38% of patient images were unavailable 
for the physician when requested. Follow-on studies at other major medical centers 
(Tripler, and Brooke) revealed similar loss rates (Leckie,1992). The results of the 
questionnaire responses from physicians at Fitzsimons support Leckie’s data. The 
majority of physicians felt that images were unavailable to them between 5-20% of the 
times requested. With the digital imaging system, the majority of the physicians (29%) at 
Madigan felt that archived images were unavailable due to loss less than 5% of the times 
requested. On the other hand, many physicians at Fitzsimons (33%) felt that images were 
unavailable due to loss 5-10% of the times requested. (Appendix B) 

Although difficult to assess a cost saving, the most severe consequence of a lost 
image is delayed diagnosis and treatment, which can result in death. Some other 
unfortunate consequences include inappropriate treatment, poor quality patient care, and 
increased radiation exposure to patients where images are retaken. Legal claims for 
military dependents and retirees may result from any of the above consequences. The 
magnitude of the legal costs associated with those claims was demonstrated at Brooke 
Army Medical Center in the Winters Economic Analysis discussed in Chapter II (Winters, 
1994), | 

The results of the questionnaire validate previous studies indicating that digital 
imaging technology decreases the number of images lost within a facility. The majority of 


physicians at Madigan felt that images were unavailable less than 5% of the times 
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requested, a significant improvement over Leckie’s findings (38% loss rate). Specific 
benefits are difficult to determine due to the qualitative nature of the consequences of lost 


images. 


c. DOES DIGITAL IMAGING IMPROVE PHYSICIAN EFFICIENCY BY 
REDUCING TIME TO RETRIEVE IMAGES? 


A physician’s time is very valuable. Finding ways to improve physician efficiency 
will generate qualitative as well as quantitative savings in military medical treatment 
facilities. The frequency of response to question 2, “On the average how much time does 
it take to retrieve an image from your storage location?” shows a significant difference 
between facilities. The largest percentage of physicians at Madigan (47%) reported that 
archival images are available to them within 10 minutes. By comparison, 42% of the 
physicians at Fitzsimons responded that it takes more than 2 hours to retreive an image 
from the Radiology records storage area. (Appendix B) 

An analysis of the clinic outpatient visits at Madigan shows an additional 26,000 
outpatient visits between 1991, when Madigan used conventional film technology, and 
1993, when Madigan used digital technology (see Figure 4-3). It is unclear what caused 
this additional outpatient workload. The number of assigned physicians has not increased 
but the in-house staff is treating more patients on an outpatient basis. The improved 
physician efficiency may be due in part to the ability to retrieve archived images faster. 
Conversely, outpatient visits at Fitzsimons have decreased dramatically over the same time 
period. The cause of this decrease is also unclear, but it could be related to the decreasing 
patient population at Fitzsimons Army Medical Center due to downsizing at nearby 
military bases. Recently the hospital was included on the Base Realignment and Closure 
list. This thesis assumes that using conventional technology has not caused the decrease in 
outpatient visits at Fitzsimons. 

Detailed data were unavailable to draw specific conclusions about physician 


efficiency as related to the digital imaging system. In the future, the teleradiology 
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initiative, “Project Seahawk”, hopes to demonstrate improved physician efficiency through 


the use of remote consultations and radiologic diagnoses. 
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Figure 4-3. Number of Outpatient Visits 
The results of the questionnaire clearly show that physicians can retrieve archived 
images faster under a digital technology imaging system. It remains unclear as to whether 
this time saving tool improves physician efficiency because data are not available to 


validate how physicians at Madigan use their time. 


D. MODIFIED LIFE-CYCLE COST CALCULATION 


To calculate cost effectiveness, one must begin by examining the overall system 
cost. For the purposes of this research, the system cost only includes the investment 
equipment cost and maintenance cost. Direct costs such as film, processors, and film 
processing chemicals are not included in the system cost because Madigan is running a 


redundant system using both digital imaging and a film back-up. Operations costs are also 
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not considered due to the uncertainty associated with staffing when Madigan goes to a 
strictly filmless radiology department. To date, it is unknown whether digital imaging 
technology will reduce staffing requirements. Therefore operations costs are excluded 
from the analysis. 

The MDIS system is a joint Service acquisition project with the Army Medical 
_ Department currently acting as the lead agent. An indefinite requirements contract was 
selected for this procurement because the project office wanted to assess the commercial 
product before deciding to place more orders against the contract. With this fixed-price 
type of contract, the Government can place orders without discussing price, quantity, or 
performance period with the contractor (Contingency Contracting, 1993). If the 
Government is not satisfied with the product performance, there is no obligation to place 
any more delivery orders. In this way, the project office can control how many systems 
will be fielded to DoD medical treatment facilities. 

To date, the project office has issued 11 delivery orders against the indefinite 
delivery order contract. The contract was signed in September 1991 and was supposed to 
expire in September 1994; however, the contract was recently extended for one year. 
Progress payments are made to the contractor based on a milestone concept. There are 
three phases for the total installation and each phase has four major steps, including 
receipt of equipment, installation of equipment, system integration, and total integration. 
Each of these major steps has specific milestones. When a milestone is 80% complete, an 
80% payment is made to the contractor. The remaining 20% is paid after each milestone 
is completed and the Government conducts Acceptance Testing.(Romlein, 1994) 

The total investment costs including installation, integration, and acceptance 


testing by phases at MAMC are shown below: 





Table 4-1. Total Investment Costs at MAMC (Winters, 1995) 
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The MDIS contract contains a one year warranty clause from the day of 
Government acceptance. None of the phases at the MAMC site have been completed. 
Therefore, the warranty period has not started on any phase of the MAMC 
implementation process. The maintenance costs begin accruing when the warranty period 
expires and will be phased in upon completing Government acceptance testing. The 
maintenance costs negotiated previously are shown below in Table 4-2. Maintenance 
costs have been separated by phases, spokes (satellite medical treatment facilities), and 


specific equipment lines (optimized workstations and RIS terminals). 





Total Maintenance Cost $ 706,178 


Table 4-2. Estimated Annual Maintenance Costs 
for MAMC MDIS System (Winters, 1995) 


The cost of the film-based imaging system at Fitzsimons is based on the property 
records for each piece of equipment. The Radiology hand receipt is separated into eight 
separate documents with a corresponding dollar value of $14,796,653. The system used 
to account for the equipment does automatic depreciation calculations in accordance with 
applicable regulations.(O’Bailey, 1995) The current dollar value is not used in the 
modified life-cycle cost spreadsheet because the $14 million represents a sunk cost. To 
maintain the current system, costs for capital investment equipment, also called extended 
Medical Care Support Equipment (MEDCASE) requirements average $4.0 million. 
(Mericie, 1995). The MEDCASE manager provided the 1993 requirements as shown in 
Table 4-3. The table’s figures represents a high end year, with costs close to $5 million. 
In the two subsequent years, the MEDCASE funding has dropped significantly because 


Fitzsimons is a candidate for Base Closure. This thesis does not use information from this 











period and assumes that MEDCASE requirements would remain the same if not 


threatened by base closure. 


47,369 


Table 4-3. Fitzsimons New Equipment Costs for 1993 





Maintenance of the Radiology film-based equipment is accomplished by the 
biomedical repair section. The only complete year of maintenance costs the facility could 
provide showed an annual cost of $469,768 (McCune, 1995). The Chief of Biomedical 


Maintenance, Fitzsimons breaks out the maintenance costs for 1994 as follows: 







Man-hour Cos 30,312 
Repair Parts 34,246 
Contract Costs 405,210 
Total $469,768 


Table 4-4. Fitzsimons Annual Maintenance Costs 





The modified life-cycle cost spreadsheets are shown in Table 4-5, for each facility. 
For the purposes of this study, only the investment costs and maintenance costs are 
included in the calculations. The discount rate used is 5.5% in real dollars in accordance 


with Office of Management and Budget Circular Number A-94 (February, 1994). 
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MAMC Life Cycle Cost (minus Operations ) 


Total investment Expense 9,863,478 


Maintenance Cost Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Discounted $ 


Phase | subsystem 363,309 363,309 326,978 326,978 290,647 290,647 290,647 
Phase I! subsystem 155,463 155,463 139,916 139,916 124,370 124,370 124,370 
Phase ill subsystem 110,147 410,147 99,132 99,132 99,132 88,117 88,117 
Fairchild 44,960 44,960 40,873 40,873 36,786 36,786 36,786 
McChord 26,314 26,314 23,922 23,922 21,529 21,529 21,529 
RIS Terminals 4,356 4,356 4,356 4.356 4,356 4,356 4,356 
Optimized workstation 1,629 1,629 4,629 1,629 1,629 1,629 1,629 


Total Maintenance Cost 706,178 706,178 636,806 636,806 578,449 567,434 567,434 


FAMC Life Cycle Cost (minus Operations) Net Present Value MAMC 
Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 
New Equipment Cost 4,000,000 3,791,469 3,593,810 3,406,455 3,228,867 3,060,537 2,900,983 2,749,747 
Maintenance 
Man-hours 30,312 28,732 27,234 25,814 24,468 23,193 21,984 20,838 
Parts 40,688 38,567 36,556 34,650 32,844 31,132 29,509 27,370 
Repair Contracts 405,210 384,085 364,062 345,082 327,092 310,040 293,877 278,556 
Total 476,210 451,384 427,852 405,547 384, 405 364,365 345,369 327 364 


Net Present Value FAMC 


Table 4-5. Modified Life Cycle Cost 





The net present value for Madigan (MAMC) is significantly lower because the 
digital imaging system is new. The equipment replacement figures reflect that several 
items will be covered under a warranty for one year within the time horizon of the 
spreadsheet (Romlein, 1995). The total modified life-cycle cost figure for each system will 


be used as the denominator in the final cost effectiveness calculation. 


E. COST EFFECTIVENESS CALCULATION 


The first step in the TRI-TAC methodology for determining cost effectiveness is to 
specify the utility values for each measure of effectiveness. The baseline figures will 
simply be 5 for each MOE as shown in Table 3-1. The more difficult task is to assign 
values to the alternative, the digital imaging system. Retakes showed only a slight 
difference when comparing actual rates reported in the questionnaire by the radiology 
department. Hence, the utility assignment for this MOE will be a 7. This corresponds to 
the lower end middle range score on the utility assignment criteria. The second MOE, 
number of lost images, was a significant attribute in both the questionnaire responses and 
previous studies on lost images with film-based technology at Madigan (Leckie, 1992). 


The utility value assigned to lost images is a 9. The third MOE, time to retrieve archival 








images, showed a significant difference in both the questionnaire and in the increased 
number of outpatient visits. Therefore it also has a utility assignment of 9. 

The next step is to weight each of the figures of merit for each measure of 
effectiveness. The first MOE is retakes; a weight of .2 is assigned for each figure of merit 
because they all have approximately equal importance to the overall system. The second 
MOE, lost images, has availability weighted most heavily with .3, dependability, capability, 
and useability weighted equally (.2) and maintainability the lowest weight (.1). The third 
MOE is time to retrieve historical images. Availability, dependability, and capability are all 
weighted equally, at .3 , and a lower weight is given to maintainability and useability (.05) 
because these have little impact on this measure. A summary of the utility scores and 


weights are shown in Table 4-6 below: 





Table 4-6. Assignment of Utility and Weights 


Next a total score for each figure of merit is calculated using the formula: 


> W; x Uji 
FOM: = SS 
DW 


where, | 
FOMj: FOM for the ith alternative 
Wj: weight for the jth MOE 
Uji: utility assigned to the i" alternative with respect to the a 


MOE 
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Table 4-7 summarizes each FOM total for each alternative. The results are scored 
according to their relative importance and the final overall effectiveness score (E) is shown 
for each alternative. The final number represents the overall effectiveness divided by the 


life-cycle cost for each alternative. 





Table 4-7. Cost Effectiveness Calculation 


The results of the cost effectiveness calculation favor the digital imaging system by 
a factor of approximately 2.5 to 1. In terms of effectiveness per dollar invested, the digital 
system achieves over twice the benefit of the conventional film-based system. The primary 
reason for this difference is that the life cycle cost is half as much for a digital system when 
only maintenance and investment costs are considered. This ratio is a conservative 
estimate because salvage value of the system at the end of the time horizon was not 
considered. If salvage value was included it would increase the cost difference between 
the two systems, favoring the digital system. As the operational proficiency with digital 
imaging improves at Madigan, and reliance on a film back-up system dissipates, the 


operations costs will decrease significantly. 
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V. CONCLUSION 


A. GENERAL 


The MDIS system is a technological leap into the future. The MDIS system, when 
complete, will meet the goal of the Government’s healthcare reform by improving 
healthcare quality and providing a more cost effective imaging system. The decreasing 
Federal budget and increased demand for quality warrants using cost effective state-of- 
the-art technology in the Department of Defense. The purpose of this study was to 
validate claims of improvements and extend previous studies of the system. The results 
are based on three measures of effectiveness: image retakes, lost images, and time to 
retrieve archived images. The two methods used to collect the data were questionnaires, 
completed by physicians who work under two different imaging technologies, and 
interviews with specialists in logistics, financial management, patient administration, and 
information management at each facility. 

The sites chosen for the study included Madigan Army Medical Center and 
Fitzsimons Army Medical Center. Madigan is the first medical treatment facility within 
the Department of Defense to receive the MDIS system. It was compared with the 
baseline alternative, Fitzsimons, which represents a typical film-based imaging system. 
Although each facility supports a different population, they are similar in terms of 
regional area mission responsibilities. 

The questionnaire results were consolidated and recorded as frequency data. 
Interview data were used in the spreadsheets, tables, and graphs. A modified life cycle 
- cost was developed for each system using capital investment and maintenance costs. A 
utility function was developed for each MOE based on quantitative and qualitative 


evaluation of the data. Finally an effectiveness score was calculated for each imaging 


system. 








B. SUMMARY OF RESULTS 


The overall questionnaire results showed that physicians completing the 
questionnaire slightly favored digital imaging technology. This limited difference was not 
surprising, however. Madigan was the first operational system user. The throughput rate 
for retrieving images is not yet fast enough for the radiologist to meet the proper diagnosis 
rate. Under a film based system, a radiologist can make a diagnosis every 90 to 180 
seconds (Bender, 1994). As a result, Madigan uses digital imaging with a film-based 
back-up system. It runs a dual technology radiology imaging system. The system users’ 
expectations were too high as a result of promised capability that has yet to materialize. 
For example, the system is not yet integrated with the magnetic resonance imaging system. 
Although digital images are obtained, they are not yet networked within the MDIS system. 
Many software upgrades have been installed causing some system downtime and 
inconvenience to the users. The implementation schedule has slipped significantly due to 


system integration, software, and requirement changes. 


C. RESEARCH QUESTIONS 


The primary research question is: Are there cost effectiveness differences between 
the state-of-the-art digital imaging system and the traditional film-based imaging 
technology most frequently used by Government and civilian medical treatment facilities? 
In the final cost effectiveness calculation, the digital imaging technology provided greater 
benefits than film-based imaging technology by a ratio of 2.54 to 1. This means that the 
benefits per dollar investment for the MOEs used in this study are roughly two and a half 
times greater for digital imaging than film. The difference is large because the technology 
is more effective and it is also cheaper. This ratio is a conservative estimate because 
salvage values at the end of the seven year life span were not considered. Obviously, the 
new system would have a greater salvage value than the old system at the end of its 
expected life. Therefore, the digital imaging technology provides more than twice the 


benefit of the film-based system and is also cheaper. 
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not a significant difference in the answers provided in the questionnaire from the two sites. 
However, a trend analysis of the retake data provided by the Madigan Radiology 
Department suggest that in the future the result could become significant as technicians 
learn to use the digital system more effectively. 

The subsidiary research question that addresses differences in the number of lost 
images favored digital imaging. There was a 10% difference of questionnaire responses 
between Madigan and Fitzsimons, pertaining to lost images, that favored digital 
technology verses film. The qualitative issues associated with lost images including: 
incorrect treatment, delayed diagnosis, increased exposure as a result of retakes, and 
decreased productivity in clinics make digital imaging superior to film-based imaging. 

The third subsidiary question addressed time to retrieve archived images. The 
results of the questionnaire clearly show that physicians can retrieve archived images faster 
under a digital imaging system. It remains unclear whether this time saving tool improves 
physician efficiency. The qualitative issues associated with this MOE, such as time to 
diagnosis and treatment, strongly favor digital imaging. 

The fourth subsidiary question addresses effectiveness per investment dollar. The 
denominator in this approach is the life cycle cost of the system. The large cost 
differences between the two systems is explained by two primary factors. First, the new 
system has very little replacement equipment cost in the seven year time horizon used in 
the modified life-cycle calculation. Further, the MDIS equipment has a one year warranty 
period built into the contract, during which there are virtually no maintenance costs. 
Second, the older system requires high capital investment to replace old equipment and to 
modernize outdated equipment within the Radiology Department. The fiscal year 1993 
new equipment costs at Fitzsimons, under the MEDCASE program, were roughly $5.2 


million. However, in keeping with conservative estimates, only a $4.0 million annual 


capital equipment cost was used in the life-cycle calculation. 








D. AREAS FOR FURTHER RESEARCH 


There are many opportunities for improving the Department of Defense medical 
system under the Government’s National Healthcare Reform initiatives. The Military 
Services, in a joint effort, need to reengineer clinical practices in medicine, with 
affordability as the over-arching goal (Jenkins, 1995). Seeking technology with dual-use 
capability between civilian and military healthcare systems decreases development and 
production costs. The MDIS system is a step toward meeting the goals of these initiatives. 
This study looked at whether digital imaging improves image retake rates, decreases lost 
images, and decreases the time spent retrieving archived images. However, there are 
many other applications associated with digital imaging that are yet to be measured in 
terms of cost, benefits, and effectiveness. When all phases of the system are integrated 


within and between facilities these applications will include: 


e Computer-Aided Diagnosis 
e Real-time Image Interpretation 
e Teleradiology 


e Continuing Medical Education 


Researchers at the University of Chicago are studying computer-aided diagnoses 
for radiologists. They built a limited database of patient case files and compared the 
radiologist’s diagnoses with computer generated diagnoses. They found that the 
computer diagnosis came up with a high false negative rate. But when the computer 
provided assistance to the radiologist, there was a significant decrease in the number of 
missed diagnoses. (Kunio, 1995) Computer-aided diagnosis has tremendous potential for 
improving the quality of patient care, but cost benefit and cost effectiveness has yet to be 
determined. 

The ideal Radiology Department could interpret images in real-time. This time 
saving technique results in quicker diagnosis and treatment and may preclude unnecessary 
return visits by patients. Some improvements to the current system must be implemented 
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before this can be accomplished. One example is increased luminescence to achieve 
diagnostic quality images. Radiology Departments may be able to reassign radiologists to 
specific areas (e.g. Department of Surgery, Orthopedics, Emergency Room) to facilitate 
real-time image interpretation. 

The teleradiology portion or phase III of the MDIS implementation schedule will 
link several medical treatment facilities from all Services 1n a project called Seahawk. The 
project goals include increased efficiency through teleconferencing, a corresponding 
decrease in patient referrals to other facilities, and improved quality of patient care by 
providing faster Services. 

An application of MDIS is developing teaching files for residents and continuing 
medical education for physicians. This benefit 1s difficult to measure in terms of cost but 


increases the quality of patient care which can be quantified in future studies. 


E. SUMMARY 


The MDIS system holds promise for reengineering healthcare in the military. The 
cost effectiveness of the system is demonstrated in this as well as previous studies. The 
MDIS system overcomes many of the limitations of film-based technology, including: the 
ability to examine an image at one location at a time, efficiently transfer images from one 
location to another digitally, and overcoming the 20-40% loss rate experienced in many 
large medical treatment facilities. There are some problems with the technology which 
also must be overcome, including producing a diagnostic quality image for radiologists, 
establishing an interface architecture with commercial radiology systems such as magnetic 
resonance imaging, and establishing realistic expectations for system performance. As the 
technology matures, the third-order benefits defined by Malone and Rockart, 1991 will 


evolve making the system more effective in the future. 
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APPENDIX A 


IMAGING SURVEY 





This survey evaluates four aspects of medical imaging in your facility. 
The four areas to be assessed are: 
e Frequency and reasons for image retakes 
e Consequences of lost images 
e Reasons and frequency of lost images 
e The time to retrieve archival images 





Please take a moment to complete this survey. Your responses will be used to evaluate the costs and 
benefits of two imaging technologies. 


Circle the letter which best answers the questions below: 


1. How many times a month does your department, ward, or clinic request archival images from the records 
storage area within your facility? 


a. 0-35 

b. 36-70 

c. 71-100 

d. greater 100 
e. don’t know 


2. On the average, how much time does it take to retrieve an image from your organizations record storage 
location? 
a. 10 minutes 
b. 30 minutes 
c. 1-2 hours 
more than 2 hours 
don’t know 





eo 


3. What percent of archival images are not available when requested by a clinician? 


a. less than 5% 

b. 5-10% 

c. 10-20% 

d. greater than 20% 
e. don’t know 





4. What percent of patient images require retakes? 


less than 1% 
1-5% 

. 5-10% 

d. greater than 10% 
e. don’t know 


oop 
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5. Of the personnel listed below, place a number from 1 to 5 in the boxes to indicate the person who most 
frequently retrieves images from the image storage room? (1 indicates most frequent and a 5 indicates least 
frequent.) 


Clerk{ | Technician| | Nurse{ | Physician{ _| Other [] 


Please take a moment to write your response in the space provided: 


6. From the most serious to least serious, list the 5 most severe consequences of lost images in your department. 


(1) 
(2) 
~ GQ) 
(4) 
(5) 

















7. From the most frequent to least frequent, list the 5 most common causes of lost images within your facility. 


(1) 
(2) 
(3) 
(4) 
G) 











8. Additional Comments: 


Fill in the appropriate information: 


NAME (Optional) 





POSITION 


RANK/GRADE 


DEPT/CLINIC/WARD 





Circle your response: 
Are you willing to discuss this survey in a follow-on interview? YES NO 


If yes, to previous question please provide duty phone number in the space provided: 
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APPENDIX B 


QUESTIONNAIRE RESULTS 


MAMC FAMC 


How many times do you request an image from the storage area within your facility? 
Possible resp. 0-35 36-70 71-100 >100 Don't know Possible resp. 0-35 36-70 71-100 >100 Don't know 
QUESTION’? [A]. 8B 1c | O | ©£ | SUM QUESTION? | A | 8B | c J O | E | 
FREQUENCY | 18] 17,475 | 1 | 9 | 08  (rRequency | 7 [| 8 | 2 [7 {2 
PROBABILITY | 031] 019 | 009 | 026 | O16 | 1  \PROBABLITY | O47 { O22 [ 006 [019 [00s [7 | 
IcCUMPROB. [031] 050 | 059 | 084 | 100 | cUMproe | o47 [069 [ 076 [094 [100 | 


On the average, how much time does it take to retrieve an image from your storage location? 
Possible resp. 10 min 30 min 1-2 hrs >2hrs Don't know Possible resp. 10 min 30 min 1-2 hrs >2hrs Don't know 


QUESTION? [A] B Jc] D> | © | SUM aquestON2 | A J] 8 | © | Oo | © | SUM 
FREQUENCY | 27] 17 [4 | 1_ | 9 | 98 |pRequency_|_7_| 7 | 9 | | 4 | 36 
PROBABLITY | 047] 029] 007] 002] 0.16 | 1 [PROBABILITY |003 | O19 | ov | o42 | O77 | 1 
CUMPROS | 047] 076] 083] 084 | 100_| Joumpros | 003 | 022 | oa7 { os [| 100 | | 


What percent of archival images are not available (lost) when requested by a clinician? 
Possible resp. <5% 5-10% 10-20% >20% Don'tknow Possible resp. <5% 5-10% 10-20% >20%  ODon'tknow 































QUESTIONS [A] BJ Co TSCOCOCSCdSCOCSE:C*sdSC(sS‘SU=dqQUESTIONNS | A | 8 yc | O J © | SUM 
FREQUENCY | 1716. 0S CdSC*‘“‘(§]O FREQUENCY [6 | 2 | 8 {| 6 | 4 | 3% 
PROBABILITY. | 029 [0.10] 0.17 | 0.16 {028 | 1 [PROBABILITY 


CUMPROB [029] 040] 0s? | 072 | 100] JcUMPRop | 017 | 080 | 072 | 009 | 10 | | 


What percent of patient images require retakes? 
Possible resp. <1% 1-5% 5-10% >10% Don't know Possible resp. <1% 1-5% §-10% >10% Don't know 
"—A_] 8]. Cc [oO J] © | SUM jQuesrionés [ A 7 68 [| Cc yO UTE 7 sum 
FREQUENCY [7 |21 +41 2~SO«SsC<CSN;*“‘)!sSSCd]SCtCPReEQUeNcy | 4 [it [a7 [3 7 
PROBABILITY | 012] 030] 020 | 007 | 076 | 1  |pRoBABLITy | O47 | 031 [O37 [008 [og [77 | 
CUMPROB {012 | 048 | 078 | 084 | 100 | |cuUMPROB | O17 | 042 [072 [oat [700 [7 


Of the personnel listed below, who retrieves images most often? 

































moresponse | 26 | 28 {33 | 12 __| 
ae ce Saas (a ay a a Ea" 
ge ee ee 
Pee eR ER. (ET a] A a 
en 
Pe 





QUESTION 6 From the most serious to least serious, list the 5 most QUESTION 7 From the most frequent to least frequent, list the 5 most 








severe consequences of lost images in your department. common causes of lost images within your facility 
Response 
_MAMC | FAMC | 
elayed diagnosis and treatment 
etake image 13 19 MOISdown 
ncrease time and money Mrongjacket 


bee” | 


atient dissatisfaction 
hysician frustration 

nable to compare studies 
oor patient care 

egal cost 


U 


Poorfim | +i| ~*~ id. 0 ~=i* 
Physiciantake—SSSCSC~C~C~—~idTSC‘ CT CC 
Misplaced SCCOCS™C™~C—~—CiTST 
Consultantstake 
ost in MDIS 3 

atients Take 

isfile 

linic takes 

-fays not read by radiologist 
iim made into teaching file 
ech does not put on system 
ot ordered 

oesn't happen 

onfusion 


OQ] =] U 





ncreased exposure 


ncrease hospital stay 
ncrease clinic watt 
ncrease patient risk 


—+ 


| 


Wrong treatment 
cor institutional image 
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a. = 
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a 5 
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ost in transit 
islabeled 
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QUESTION 8 Additional Comments 




















1. Most of the images are on computer so they are not lost. 
2. With the MDIS system, film retrieval is far easier than 
facilities without MDIS. 

3. You should define your terms before asking questions. 
4. The system (MDIS) is not reliable. 

5. Need longer times before station signs off during clinic hours. 
6. Most of the time there are no serious problems with MDIS 
since retrieval of the image is available all the time. 

7. The only way to get the important studies (CXR, bone films) 
is to ask the patient to bring the unread wet film back, and 

hand carry to radiologist for diagnosis. 

8. We need an MDIS terminal in oncology. 

9. Still patiently awaiting MDIS for HEME/Onc section. 

40. IfMDIS is the answer to this, we are extremely frustrated 

by still not having a single MDIS workstation in the clinic 
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1. | really would like to retrieve films on weekends, is this possible? 
2. Easy access to timely written reports would be more helpful than 
access to film. 

3. Mortise views in cast are most common cause of retakes. 

4. Often times quality of images is not adequate- rotated views 
accepted by technicians require reordering- cause clinic delays. 











The following bar charts show a comparison of each medical treatment facility’s questionnaire 


results. 


What percent of patient images require retakes? 


Number of Retakes 


alas 


cet 
vn 


esahatetatceseetanaeestor as aero feteereananespZanensnstmnetete 
Ronanaanrn tre conn uous snsnenae renee 


Percent Response 


ne os 


1-5% 5-10% 
Frequency of Retakes 








How Often Images are Unavailable (Lost) 












Percent Response 


. 
oR oaths cae e's. ee a 


<5% 5-10% 10-20% 





Loss Rate 


What percent of archival images are not available (lost) when requested by a physician? 





Time to Retrieve an Archived Image 


Percent Response 
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36-70 71-100 
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APPENDIX C 
INTERVIEW INSTRUCTIONS 


My name is Major Tracey Syvertson, I am an Army Medical Service Corps officer 
assigned to the Naval Postgraduate School. I am doing a study which compares two 
different imaging technologies at Madigan and Fitzsimons Army Medical Centers. I 
would like to ask you specific questions that will be used in my research project. Will you 
assist me in this regard? If you would like to preserve your anonimity, I will withhold 


your name from the study and quote only your title. 
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